Skip to content

Seeding for build_vset, two rounds of bug fixes, Python 3.13, new logo - #60

Merged
csinva merged 6 commits into
masterfrom
bugfix-and-seeding
Oct 15, 2026
Merged

csinva merged 6 commits into
masterfrom
bugfix-and-seeding

Conversation

@csinva

@csinva csinva commented Oct 15, 2026

Copy link
Copy Markdown
Member

Summary

  • Seeding: build_vset(..., reps=n, seed=s) gives each vfunc its own random_state from np.random.SeedSequence(s), so repeated perturbations (e.g. bootstrap samples) are reproducible. The README example uses it.
  • Bug fixes from two review rounds, each with a regression test in tests/test_bugfixes.py: filter_vset_by_metric, dict_to_df(param_key=...), perturbation_stats (precedence bug, tuple group keys, 2-D split, unsupported funcs), PCSPipeline.run/generate_names, MLflow tracking (params, metric names, unclosed runs), caching (stale results, lambdas, by-value cache keys), Vset construction checks, lazy async chaining, Subkey hashing.
  • CI: test on Python 3.10, 3.12 and 3.13 (3.11 dropped).
  • Docs: index ends at References, Contributing removed from the guides list, new drop-ring logo, wordmark and favicon (logo_vflow_straight.png kept for external links).

Testing

93 tests pass locally on 3.12 (CI command: random order, xdist, coverage) and on 3.13; ruff, black and isort are clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_013Xy8Dg7fAqocHGzMqVCiSK

csinva and others added 6 commits October 15, 2026 16:56
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Xy8Dg7fAqocHGzMqVCiSK
Seeding: build_vset(..., reps=n, seed=s) gives each vfunc its own
random_state drawn from np.random.SeedSequence(s), so repeated
perturbations (e.g. bootstrap samples) are reproducible.

Bug fixes (each has a regression test in tests/test_bugfixes.py):
- filter_vset_by_metric kept nothing for build_vset output without
  param_dict or with reps, double-counted configurations across
  upstream perturbations, and could not filter a filtered Vset
- dict_to_df(param_key=...) mixed up parameters across rows and split
  values containing '='
- perturbation_stats: operator-precedence bug, string/callable func,
  prefix applied to default columns
- PCSPipeline.run did not chain steps; generate_names crashed
- MLflow tracking: params only logged in some cases, runs never closed,
  non-numeric outputs (fit/predict) crashed logging
- caching served stale results after a function was redefined and
  failed on lambdas; cache now keyed on a fingerprint of the inputs
- Vset accepted bad containers and duplicate keys, ignored async
  vfuncs given as a dict, and dict keys were not turned into Subkeys
- predict/predict_proba on a Vset without a predictor now raise
- lazy async outputs feeding a lazy sync Vset were not resolved
- Subkey hash disagreed with equality
- cum_acc_by_uncertainty gave an unclear error on class predictions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Xy8Dg7fAqocHGzMqVCiSK
- caching: key the cache on a by-value hash of the inputs (joblib.hash,
  without the lineage entry) so equal copies, e.g. outputs loaded back
  from the cache, hit it; add the source of module-level functions so
  editing one does not serve stale results
- MLflow: metric names from build_vset tuple keys are sanitized; runs
  are closed even if logging fails; runs are keyed by full subkeys so
  equal values from different Vsets are not merged
- PCSPipeline.run passed Vfunc objects on from an already-fitted step
- vfunc_keys that compare equal (1 and 1.0) were not caught as duplicates
- perturbation_stats on array entries: tuple group keys (build_vset
  models, predict_proba with uncertainty grouped by model) crashed,
  split=True crashed on 2-D entries (now out<i>_<j> columns), and
  unsupported funcs were silently ignored (now raise)
- build_vset: partials of one function get distinct names, and distinct
  functions or values that share a key raise instead of being dropped

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Xy8Dg7fAqocHGzMqVCiSK
The docs header uses the wordmark, the favicon is the drop ring, and the
README points at the new wordmark. logo_vflow_straight.png is kept since
external pages may link to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Xy8Dg7fAqocHGzMqVCiSK
@codecov

codecov Bot commented Oct 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.37984% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.46%. Comparing base (9bad634) to head (a338005).

Files with missing lines Patch % Lines
vflow/vset.py 85.71% 11 Missing and 2 partials ⚠️
vflow/helpers.py 83.82% 8 Missing and 3 partials ⚠️
tests/test_bugfixes.py 99.22% 2 Missing ⚠️
tests/test_seeding.py 96.15% 2 Missing ⚠️
vflow/pipeline.py 90.90% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #60      +/-   ##
==========================================
+ Coverage   89.47%   93.46%   +3.98%     
==========================================
  Files          12       14       +2     
  Lines        1454     1897     +443     
  Branches      177      211      +34     
==========================================
+ Hits         1301     1773     +472     
+ Misses         99       79      -20     
+ Partials       54       45       -9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@csinva
csinva merged commit f6218af into master Oct 15, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant